{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Wide & Deep Recommender Demo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wide and Deep Learning Model, proposed by Google in 2016, is a DNN-Linear mixed model. Wide and deep learning has been used for Google App Store for their app recommendation.\n",
    "\n",
    "In this tutorial, we use Recommender API of Analytics Zoo to build a wide linear model and a deep neural network, which is called Wide&Deep model, and use optimizer of BigDL to train the neural network. Wide&Deep model combines the strength of memorization and generalization. It's useful for generic large-scale regression and classification problems with sparse input features (e.g., categorical features with a large number of possible feature values)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intialization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "import necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "from zoo.models.recommendation import *\n",
    "from zoo.models.recommendation.utils import *\n",
    "from zoo.common.nncontext import init_nncontext\n",
    "import os\n",
    "import sys\n",
    "import datetime as dt\n",
    "\n",
    "import matplotlib\n",
    "matplotlib.use('agg')\n",
    "import matplotlib.pyplot as plt\n",
    "%pylab inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initilaize NN context, it will get a SparkContext with optimized configuration for BigDL performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "sc = init_nncontext(\"WideAndDeep Example\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download and read movielens 1M rating data, understand the dimension."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1000209, 3)\n",
      "(1, 6040, 1, 3952, array([1, 2, 3, 4, 5]))\n"
     ]
    }
   ],
   "source": [
    "from bigdl.dataset import movielens\n",
    "movielens_data = movielens.get_id_ratings(\"/tmp/movielens/\")\n",
    "min_user_id = np.min(movielens_data[:,0])\n",
    "max_user_id = np.max(movielens_data[:,0])\n",
    "min_movie_id = np.min(movielens_data[:,1])\n",
    "max_movie_id = np.max(movielens_data[:,1])\n",
    "rating_labels= np.unique(movielens_data[:,2])\n",
    "\n",
    "print(movielens_data.shape)\n",
    "print(min_user_id, max_user_id, min_movie_id, max_movie_id, rating_labels)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform ratings into dataframe, read user and item data into dataframes. Transform labels to zero-based since the original labels start from 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "sqlContext = SQLContext(sc)\n",
    "from pyspark.sql.types import *\n",
    "from pyspark.sql import Row\n",
    "\n",
    "Rating = Row(\"userId\", \"itemId\", \"label\")\n",
    "User = Row(\"userId\", \"gender\", \"age\" ,\"occupation\")\n",
    "Item = Row(\"itemId\", \"title\" ,\"genres\")\n",
    "\n",
    "ratings = sc.parallelize(movielens_data)\\\n",
    "    .map(lambda l: (int(l[0]), int(l[1]), int(l[2])-1))\\\n",
    "    .map(lambda r: Rating(*r))\n",
    "ratingDF = sqlContext.createDataFrame(ratings)\n",
    "\n",
    "users= sc.textFile(\"/tmp/movielens/ml-1m/users.dat\")\\\n",
    "    .map(lambda l: l.split(\"::\")[0:4])\\\n",
    "    .map(lambda l: (int(l[0]), l[1], int(l[2]), int(l[3])))\\\n",
    "    .map(lambda r: User(*r))\n",
    "userDF = sqlContext.createDataFrame(users)\n",
    "\n",
    "items = sc.textFile(\"/tmp/movielens/ml-1m/movies.dat\")\\\n",
    "    .map(lambda l: l.split(\"::\")[0:3])\\\n",
    "    .map(lambda l: (int(l[0]), l[1], l[2].split('|')[0]))\\\n",
    "    .map(lambda r: Item(*r))\n",
    "itemDF = sqlContext.createDataFrame(items)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Join data together, and transform data. For example, gender is going be used as categorical feature, occupation and gender will be used as crossed features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------+------+-----+------+---+----------+--------------+------+----------+\n",
      "|itemId|userId|label|gender|age|occupation|         title|genres|age-gender|\n",
      "+------+------+-----+------+---+----------+--------------+------+----------+\n",
      "|    26|  3391|    3|     2| 18|         4|Othello (1995)|     5|        24|\n",
      "|    26|  1447|    4|     2| 18|         4|Othello (1995)|     5|        24|\n",
      "|    26|  5107|    3|     1| 45|         0|Othello (1995)|     5|         5|\n",
      "|    26|  2878|    3|     1| 50|        20|Othello (1995)|     5|        47|\n",
      "|    26|  1527|    1|     2| 18|        10|Othello (1995)|     5|        24|\n",
      "+------+------+-----+------+---+----------+--------------+------+----------+\n",
      "only showing top 5 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from pyspark.sql.functions import col, udf\n",
    "\n",
    "gender_udf = udf(lambda gender: categorical_from_vocab_list(gender, [\"F\", \"M\"], start=1))\n",
    "bucket_cross_udf = udf(lambda feature1, feature2: hash_bucket(str(feature1) + \"_\" + str(feature2), bucket_size=100))\n",
    "genres_list = [\"Crime\", \"Romance\", \"Thriller\", \"Adventure\", \"Drama\", \"Children's\",\n",
    "      \"War\", \"Documentary\", \"Fantasy\", \"Mystery\", \"Musical\", \"Animation\", \"Film-Noir\", \"Horror\",\n",
    "      \"Western\", \"Comedy\", \"Action\", \"Sci-Fi\"]\n",
    "genres_udf = udf(lambda genres: categorical_from_vocab_list(genres, genres_list, start=1))\n",
    "     \n",
    "allDF = ratingDF.join(userDF, [\"userId\"]).join(itemDF, [\"itemId\"]) \\\n",
    "        .withColumn(\"gender\", gender_udf(col(\"gender\")).cast(\"int\")) \\\n",
    "        .withColumn(\"age-gender\", bucket_cross_udf(col(\"age\"), col(\"gender\")).cast(\"int\")) \\\n",
    "        .withColumn(\"genres\", genres_udf(col(\"genres\")).cast(\"int\"))\n",
    "allDF.show(5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Speficy data feature information shared by the WideAndDeep model and its feature generation. Here, we use occupation gender for wide base part, age and gender crossed as wide cross part, genres and gender as indicators, userid and itemid for embedding.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket_size = 100\n",
    "column_info = ColumnFeatureInfo(\n",
    "            wide_base_cols=[\"occupation\", \"gender\"],\n",
    "            wide_base_dims=[21, 3],\n",
    "            wide_cross_cols=[\"age-gender\"],\n",
    "            wide_cross_dims=[bucket_size],\n",
    "            indicator_cols=[\"genres\", \"gender\"],\n",
    "            indicator_dims=[19, 3],\n",
    "            embed_cols=[\"userId\", \"itemId\"],\n",
    "            embed_in_dims=[max_user_id, max_movie_id],\n",
    "            embed_out_dims=[64, 64],\n",
    "            continuous_cols=[\"age\"])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform data to  RDD of Sample. We use optimizer of BigDL directly to train the model, it requires data to be provided in format of RDD(Sample). A Sample is a BigDL data structure which can be constructed using 2 numpy arrays, feature and label respectively. The API interface is Sample.from_ndarray(feature, label).  Wide&Deep model need two input tensors, one is SparseTensor for the Wide model, another is a DenseTensor for the Deep model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "rdds = allDF.rdd\\\n",
    ".map(lambda row: to_user_item_feature(row, column_info))\\\n",
    ".repartition(4)\n",
    "trainPairFeatureRdds, valPairFeatureRdds = rdds.randomSplit([0.8, 0.2], seed= 1)\n",
    "valPairFeatureRdds.persist()\n",
    "train_data= trainPairFeatureRdds.map(lambda pair_feature: pair_feature.sample)\n",
    "test_data= valPairFeatureRdds.map(lambda pair_feature: pair_feature.sample)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Create the Wide&Deep model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Analytics Zoo, it is simple to build Wide&Deep model by calling WideAndDeep API. You need specify model type, and class number, as well as column information of features according to your data. You can also change other default parameters in the network, like hidden layers. The model could be fed into an Optimizer of BigDL or NNClassifier of analytics-zoo. Please refer to the document for more details. In this example, we demostrate how to use optimizer of BigDL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "creating: createZooKerasInput\n",
      "creating: createZooKerasInput\n",
      "creating: createZooKerasInput\n",
      "creating: createZooKerasInput\n",
      "creating: createZooKerasSparseDense\n",
      "creating: createZooKerasFlatten\n",
      "creating: createZooKerasSelect\n",
      "creating: createZooKerasEmbedding\n",
      "creating: createZooKerasFlatten\n",
      "creating: createZooKerasFlatten\n",
      "creating: createZooKerasSelect\n",
      "creating: createZooKerasEmbedding\n",
      "creating: createZooKerasFlatten\n",
      "creating: createZooKerasMerge\n",
      "creating: createZooKerasDense\n",
      "creating: createZooKerasDense\n",
      "creating: createZooKerasDense\n",
      "creating: createZooKerasDense\n",
      "creating: createZooKerasMerge\n",
      "creating: createZooKerasActivation\n",
      "creating: createZooKerasModel\n",
      "creating: createZooWideAndDeep\n"
     ]
    }
   ],
   "source": [
    "wide_n_deep = WideAndDeep(5, column_info, \"wide_n_deep\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create optimizer and train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "creating: createAdam\n",
      "creating: createZooKerasSparseCategoricalCrossEntropy\n",
      "creating: createZooKerasSparseCategoricalAccuracy\n"
     ]
    }
   ],
   "source": [
    "wide_n_deep.compile(optimizer = \"adam\",\n",
    "                    loss= \"sparse_categorical_crossentropy\",\n",
    "                    metrics=['accuracy'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp_log_dir = create_tmp_path()\n",
    "wide_n_deep.set_tensorboard(tmp_log_dir, \"training_wideanddeep\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the network. Wait some time till it finished.. Voila! You've got a trained model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization Done.\n",
      "CPU times: user 67.5 ms, sys: 24.2 ms, total: 91.6 ms\n",
      "Wall time: 3min 16s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Boot training process\n",
    "wide_n_deep.fit(train_data,\n",
    "                batch_size = 8000,\n",
    "                nb_epoch = 10,\n",
    "                validation_data = test_data)\n",
    "print(\"Optimization Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prediction and recommendation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zoo models make inferences based on the given data using model.predict(val_rdd) API. A result of RDD is returned. predict_class returns the predicted label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 2, 2, 4, 1]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = wide_n_deep.predict(test_data)\n",
    "results.take(5)\n",
    "\n",
    "results_class = wide_n_deep.predict_class(test_data)\n",
    "results_class.take(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the Analytics Zoo, Recommender has provied 3 unique APIs to predict user-item pairs and make recommendations for users or items given candidates.\n",
    "Predict for user item pairs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UserItemPrediction [user_id: 5305, item_id: 26, prediction: 4, probability: 0.610632181168]\n",
      "UserItemPrediction [user_id: 1150, item_id: 26, prediction: 2, probability: 0.393449008465]\n",
      "UserItemPrediction [user_id: 4294, item_id: 26, prediction: 2, probability: 0.276259899139]\n",
      "UserItemPrediction [user_id: 5948, item_id: 26, prediction: 4, probability: 0.508404672146]\n",
      "UserItemPrediction [user_id: 3825, item_id: 26, prediction: 1, probability: 0.456133008003]\n"
     ]
    }
   ],
   "source": [
    "userItemPairPrediction = wide_n_deep.predict_user_item_pair(valPairFeatureRdds)\n",
    "for result in userItemPairPrediction.take(5): print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recommend 3 items for each user given candidates in the feature RDDs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UserItemPrediction [user_id: 4904, item_id: 1221, prediction: 5, probability: 0.873028695583]\n",
      "UserItemPrediction [user_id: 4904, item_id: 919, prediction: 5, probability: 0.860194385052]\n",
      "UserItemPrediction [user_id: 4904, item_id: 2762, prediction: 5, probability: 0.855066776276]\n",
      "UserItemPrediction [user_id: 1084, item_id: 1213, prediction: 5, probability: 0.525161027908]\n",
      "UserItemPrediction [user_id: 1084, item_id: 50, prediction: 5, probability: 0.517793118954]\n"
     ]
    }
   ],
   "source": [
    "userRecs = wide_n_deep.recommend_for_user(valPairFeatureRdds, 3)\n",
    "for result in userRecs.take(5): print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recommend 3 users for each item given candidates in the feature RDDs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UserItemPrediction [user_id: 2244, item_id: 1084, prediction: 5, probability: 0.879560172558]\n",
      "UserItemPrediction [user_id: 5582, item_id: 1084, prediction: 5, probability: 0.789601504803]\n",
      "UserItemPrediction [user_id: 1835, item_id: 1084, prediction: 5, probability: 0.781116306782]\n",
      "UserItemPrediction [user_id: 4511, item_id: 3764, prediction: 4, probability: 0.537709891796]\n",
      "UserItemPrediction [user_id: 5080, item_id: 3764, prediction: 4, probability: 0.464907348156]\n"
     ]
    }
   ],
   "source": [
    "itemRecs = wide_n_deep.recommend_for_item(valPairFeatureRdds, 3)\n",
    "for result in itemRecs.take(5): print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Draw the convergence curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'loss')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr8AAAF1CAYAAADhgoKhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xl4lNXd//H3mT2TfYOw7wTZQaBY\nlEVRq1brUqqtPtpNtLVaaxf1sYu2tXXhsVZ/VktbW23VSl1qXap1IQV3BUFQQHZIgGxkX2c5vz8m\nRIgsCWZyT5LP67pyZTJz5pzvDAf43Cdn7ttYaxERERER6Q1cThcgIiIiItJVFH5FREREpNdQ+BUR\nERGRXkPhV0RERER6DYVfEREREek1FH5FREREpNdQ+BURiSNjzDZjzHyn6xARkRiFXxERERHpNRR+\nRURERKTXUPgVEekCxhi/MeZOY8yulq87jTH+lsdyjDHPGGMqjTF7jTHLjTGulseuNcYUGWNqjDEb\njDEnOftKRES6N4/TBYiI9BI3ADOByYAFngJ+DPwE+D5QCOS2tJ0JWGNMPvAdYLq1dpcxZijg7tqy\nRUR6Fq38ioh0jQuBn1trS6y1pcBNwP+0PBYC+gFDrLUha+1ya60FIoAfGGuM8Vprt1lrNztSvYhI\nD6HwKyLSNfoD2/f7eXvLfQC3A5uA/xhjthhjrgOw1m4CrgZuBEqMMX83xvRHRESOmsKviEjX2AUM\n2e/nwS33Ya2tsdZ+31o7HDgLuGbf3l5r7cPW2uNbnmuBW7u2bBGRnkXhV0SkazwC/NgYk2uMyQF+\nCvwNwBjzeWPMSGOMAaqIbXeIGmPyjTEntnwwrhFoAKIO1S8i0iMo/IqIdI1fAu8C7wNrgJUt9wGM\nAl4CaoE3gN9Za5cS2+97C1AG7AH6ANd3bdkiIj2LiX2mQkRERESk59PKr4iIiIj0Ggq/IiIiItJr\nKPyKiIiISK+h8CsiIiIivYbCr4iIiIj0Gp54dJqRkWFHjhwZj66lG6qrqyM5OdnpMiSBaE7I/jQf\npC3NCWnrUHNixYoVZdba3I70FZfw27dvX9599914dC3dUEFBAXPnznW6DEkgmhOyP80HaUtzQto6\n1Jwwxmz/ZOvD07YHEREREek1FH5FREREpNdQ+BURERGRXiMue35FREREJCYUClFYWEhjY6PTpXRb\nKSkphEIhvF7vp+5L4VdEREQkjgoLC0lNTWXo0KEYY5wup9ux1lJYWEhhYSHDhg371P1p24OIiIhI\nHDU2NpKdna3ge5SMMaSnp3fayrnCr4iIiEicKfh+Op35/in8ioiIiPRglZWV/O53vzuq555++ulU\nVla2u/2NN97IokWLjmqsrqLwKyIiItKDHS78hsPhwz73ueeeIyMjIx5lOUbhV0RERKQHu+6669i8\neTOTJ0/mhz/8IQUFBZxwwgmcddZZjB07FoCzzz6bY489lnHjxrF48eLW5w4dOpSysjK2bdvGMccc\nw6WXXsq4ceM45ZRTaGhoOOy4q1atYubMmUycOJFzzjmHiooKAO666y7Gjh3LxIkTueCCCwD473//\ny+TJk5k8eTJTpkyhpqYmTu+GzvYgIiIi0mVuevoDPtxV3al9ju2fxs/OHHfIx2+55RbWrl3LqlWr\ngNilgleuXMnatWtbz55w//33k5WVRUNDA9OnT+e8884jOzv7gH42btzII488wh/+8Ae+9KUv8fjj\nj3PRRRcdctyLL76Yu+++mzlz5vDTn/6Um266iTvvvJNbbrmFrVu34vf7W7dULFq0iHvuuYdZs2ZR\nW1tLIBD4tG/LIcVl5bchbOPRrYiIiIh0ghkzZhxw2rC77rqLSZMmMXPmTHbu3MnGjRs/8Zxhw4Yx\nefJkAI499li2bdt2yP6rqqqorKxkzpw5AFxyySUsW7YMgIkTJ3LhhRfyt7/9DY8ntg47a9Ysrrnm\nGu666y4qKytb74+HuPRcXG+JRi0ulz7ZKCIiIrLP4VZou1JycnLr7YKCAl566SXeeOMNgsEgc+fO\nPehpxfx+f+ttt9t9xG0Ph/Lss8+ybNkynn76aW6++WbWrFnDddddxxlnnMFzzz3HrFmzeOGFFxgz\nZsxR9X8kcdvzW9UQilfXIiIiItJOqamph91DW1VVRWZmJsFgkPXr1/Pmm29+6jHT09PJzMxk+fLl\nAPz1r39lzpw5RKNRdu7cybx587j11lupqqqitraWzZs3M2HCBK699lqmT5/O+vXrP3UNhxK3NeXy\nuiYyk33x6l5ERERE2iE7O5tZs2Yxfvx4TjvtNM4444wDHv/c5z7HfffdxzHHHEN+fj4zZ87slHEf\neOABLr/8curr6xk+fDh//vOfiUQiXHTRRVRVVWGt5aqrriIjI4Of/OQnLF26FJfLxbhx4zjttNM6\npYaDMdZ2/v5cf79Rdtlrb/KZ4dlHbiw9XkFBAXPnznW6DEkgmhOyP80HaaunzYl169ZxzDHHOF1G\nt1ZTU0NhYeEn3kdjzApr7bSO9BW3bQ/ldc3x6lpERERE5Kgo/IqIiIhIrxG38Lu3VuFXRERERBJL\nXMKvy8Q+8CYiIiIikkjiEn7dBspqFX5FREREJLHELfzuqvzkyZFFRERERJwUl/DrcRl2Vx3dVT9E\nRERExFkpKSkdur87iVP4hZKaJkKRaDy6FxERERE5KvHZ9uACa2FPlbY+iIiIiDjpuuuu45577mn9\n+cYbb2TRokXU1tZy0kknMXXqVCZMmMBTTz3V7j6ttfzwhz9k/PjxTJgwgUcffRSA3bt3M3v2bCZP\nnsz48eNZvnw5kUiEr371q61tf/Ob33T6a+yIuFze2GNi33dXNTIoKxiPIURERES6naufv5pVe1Z1\nap+T8yZz5+fuPOTj559/PldffTVXXHEFAEuWLOGFF14gEAjw5JNPkpaWRllZGTNnzuSss87CGHPE\nMZ944glWrVrF6tWrKSsrY/r06cyePZuHH36YU089lRtuuIFIJEJ9fT2rVq2iqKiItWvXAlBZWdk5\nL/woHXHl1xiTb4xZtd9XtTHm6sM9x+eOvWlvbSnvpDJFRERE5GhMmTKFkpISdu3axerVq8nMzGTQ\noEFYa/nf//1fJk6cyPz58ykqKqK4uLhdfb766qt8+ctfxu1207dvX+bMmcM777zD9OnT+fOf/8yN\nN97ImjVrSE1NZfjw4WzZsoUrr7yS559/nrS0tDi/4sM74sqvtXYDMBnAGOMGioAnD/cct4EZI7J5\nbGUhV540qlMKFREREenuDrdCG08LFizgscceY8+ePZx//vkAPPTQQ5SWlrJixQq8Xi9Dhw6lsfHT\nbVmdPXs2y5Yt49lnn+WrX/0q11xzDRdffDGrV6/mhRde4L777mPJkiXcf//9nfGyjkpH9/yeBGy2\n1m4/UsPPDMtme3k9YX3oTURERMRR559/Pn//+9957LHHWLBgAQBVVVX06dMHr9fL0qVL2b79iPGu\n1QknnMCjjz5KJBKhtLSUZcuWMWPGDLZv307fvn259NJL+eY3v8nKlSspKysjGo1y3nnn8ctf/pKV\nK1fG62W2S0f3/F4APNKehqmBWNc1jWEyk30dHEZEREREOsu4ceOoqalhwIAB9OvXD4ALL7yQM888\nkwkTJjBt2jTGjBnT7v7OOecc3njjDSZNmoQxhttuu428vDweeOABbr/9drxeLykpKTz44IMUFRXx\nta99jWg0tiD661//Oi6vsb2MtbZ9DY3xAbuAcdbaT2wIMcYsBBYC5ObmHnvlHQ/xp7XN3DY7iT7B\nuJxUQrqJ2traHnFeQOk8mhOyP80HaaunzYn09HRGjhzpdBndWiQSYevWrVRVVR1w/7x581ZYa6d1\npK+OrPyeBqw8WPAFsNYuBhYD5Ofn2xlTJvCntSsYO+lYxg9I70hN0sMUFBQwd+5cp8uQBKI5IfvT\nfJC2etqcWLduHampqU6X0a3V1NQQCASYMmXKp+6rI0uyX6adWx7g420P1Y2hDpYkIiIiIhIf7Qq/\nxphk4GTgifZ2nBbwArE9vyIiIiIiiaBd2x6stXVAdkc6VvgVERERibHWtuviEXJw7f2MWnvE7ZNo\nrdseGrTtQURERHqvQCBAeXl5pwa43sRaS1VVFYFAoFP6i8vljQFS9jvVmYiIiEhvNXDgQAoLCykt\nLXW6lG6rrq6OSZMmdUpfcQu/XreLoM9NjT7wJiIiIr2Y1+tl2LBhTpfRrRUUFOD1ejulr7iegDc1\n4KFK2x5EREREJEHENfyOyE3hvZ2V8RxCRERERKTd4hp+5x/Tl00ltWwtq4vnMCIiIiIi7RLX8PuZ\n4VkArN9dHc9hRERERETaJa7hNz1J5/oVERERkcQR5w+8xcKvLnEsIiIiIokgruE3xa9z/YqIiIhI\n4ohr+HW7DCl+j8KviIiIiCSEuIZfiJ3rVxe6EBEREZFE0EXhVyu/IiIiIuK8Lgi/XmqatPIrIiIi\nIs7rkpXf6gat/IqIiIiI87pm5Vd7fkVEREQkAWjPr4iIiIj0GnEPv1lBH5UNIZrCkXgPJSIiIiJy\nWHEPv2P7pxGJWtbvron3UCIiIiIihxX38DtpUAYAqwsr4z2UiIiIiMhhxT389k8PkJPiY01hVbyH\nEhERERE5rLiHX2MMw3NT2FZeF++hREREREQOK+7hF2BIVpDt5fVdMZSIiIiIyCF1TfjNDlJS00RD\ns874ICIiIiLO6ZLwOzg7GYAde7X6KyIiIiLO6ZrwmxUEoLBC4VdEREREnNMl4bdPqh+A0pqmrhhO\nREREROSguiT8Zqf4AIVfEREREXFWl4Rfv8dNepKX0lqFXxERERFxTpeEX4DcVL9WfkVERETEUV0X\nflMUfkVERETEWV268rthTw3RqO2qIUVEREREDtBl4bd/RhI1TWFufX59Vw0pIiIiInKALgu/3543\ngoDXxdpdVV01pIiIiIjIAbos/KYFvJw6Lk9XeRMRERERx3RZ+IXYld52VTYSikS7clgREREREaCL\nw++grCCRqGV3ZWNXDisiIiIiAnR1+M0MArCzQlsfRERERKTrdWn4zU2NXea4TFd6ExEREREHtCv8\nGmMyjDGPGWPWG2PWGWOOO5rBMoOx8FtZHzqap4uIiIiIfCqedrb7LfC8tfaLxhgfEDyawdKTvADs\nrWs+mqeLiIiIiHwqRwy/xph0YDbwVQBrbTNwVOnV43aRnuSlol7hV0RERES6nrH28JcbNsZMBhYD\nHwKTgBXAd621dW3aLQQWAuTm5h67ZMmSg/Z37bJ6ohaumxEgO6lLtxyLQ2pra0lJSXG6DEkgmhOy\nP80HaUtzQto61JyYN2/eCmvttI701Z7wOw14E5hlrX3LGPNboNpa+5NDPSc/P99u2LDhoI994Z7X\nWL2zkn7pAd64/qSO1CrdVEFBAXPnznW6DEkgmhOyP80HaUtzQto61JwwxnQ4/LZn6bUQKLTWvtXy\n82PA1I4Msr+ymtiZHnZX6Vy/IiIiItK1jhh+rbV7gJ3GmPyWu04itgXiqBRVNgCQlxY42i5ERERE\nRI5KezfdXgk8ZIx5H5gM/OpoB7zyxJEAZAS9R9uFiIiIiMhRadepzqy1q4AO7ac4lO+fks/uqkZe\n21TWGd2JiIiIiLSbI6dbSE/yUtWgC12IiIiISNdyLPzWN0cIRaJODC8iIiIivZRj4RfQ6q+IiIiI\ndCmFXxERERHpNRR+RURERKTXcCT89suIneN3S2ndEVqKiIiIiHQeR8Jvft9U+qT6Wbq+xInhRURE\nRKSXciT8GmOYPTqXN7eUOzG8iIiIiPRSjoRfgAEZSeytbyYStU6VICIiIiK9jGPhNyPoxVqo1ofe\nRERERKSLOBZ+M4M+ACrqm50qQURERER6GcfCb3owdrqzinqt/IqIiIhI13B85beqQSu/IiIiItI1\nnNvz23Khi4o6rfyKiIiISNdwfOW3Uh94ExEREZEu4lj4TQ14cBmo1AfeRERERKSLOBZ+XS5DRtBH\neZ3Cr4iIiIh0DcfCL8QudLFzb72TJYiIiIhIL+Jo+B2ak8y28jonSxARERGRXsTR8DssO8jOvQ0U\nVzc6WYaIiIiI9BKOht8h2ckAfOZXL2OtdbIUEREREekFHA2/kwdntN4uq9UH30REREQkvhwNvyNy\nU/jrN2YAsLG4xslSRERERKQXcDT8AuT3TQXgI4VfEREREYkzx8NvbqqfzKCXtbuqnS5FRERERHo4\nx8OvMYbPjshh+cZSfehNREREROLK8fALMCc/l+LqJjZo64OIiIiIxFFChN8xebF9vzv3NjhciYiI\niIj0ZAkRfrNT/ADsrWtyuBIRERER6ckSI/wm+wCd61dERERE4ishwm/A6yboc7O3TuFXREREROIn\nIcIvQFayT+FXREREROIqYcJvdrKPl9YVU1zd6HQpIiIiItJDJUz4DUctNY1hvvPwSqdLEREREZEe\nKmHCb01jGIBdlVr5FREREZH4SJjw+7sLpwJgjMOFiIiIiEiPlTDhd/yAdK46cSS7KhtoDkedLkdE\nREREeqCECb8Ag7KCRC0UVepKbyIiIiLS+TztaWSM2QbUABEgbK2dFo9iBmQmAbC7soFhOcnxGEJE\nREREerF2hd8W86y1ZXGrBMgMxq70VtUQiucwIiIiItJLJdS2h/QkLwCVCr8iIiIiEgfGWnvkRsZs\nBSoAC/zeWrv4IG0WAgsBcnNzj12yZEmHi2kKWy57qZ4Fo72cMdzX4edLYqqtrSUlJcXpMiSBaE7I\n/jQfpC3NCWnrUHNi3rx5Kzq6Hbe92x6Ot9YWGWP6AC8aY9Zba5ft36AlEC8GyM/Pt3Pnzu1IHfv6\nwLf0ebL6DWLu3GM6/HxJTAUFBRzNfJCeS3NC9qf5IG1pTkhbnTkn2rXtwVpb1PK9BHgSmNEpo7dh\njCE96KVa2x5EREREJA6OGH6NMcnGmNR9t4FTgLXxKigjyUtlvcKviIiIiHS+9mx76As8aWKXXvMA\nD1trn49XQRlBhV8RERERiY8jhl9r7RZgUhfUAsTO+PDejkoaQxECXndXDSsiIiIivUBCneoMID3J\nR3ldM4te2OB0KSIiIiLSwyRc+L1sznAAHn1nJ4UV9Q5XIyIiIiI9ScKF39F9U/nFF8ZR0xTm5DuW\nHfkJIiIiIiLtlHDhF+ALUwYA0BCK0ByOOlyNiIiIiPQUCRl+0wJebjtvIgDF1Y0OVyMiIiIiPUVC\nhl+A/hlJABRVNjhciYiIiIj0FAkbfvtlBADYXaXwKyIiIiKdI2HDb//02MrvrkptexARERGRzpGw\n4TfJ52ZYTjIvryvGWut0OSIiIiLSAyRs+AW45LghrNxRycaSWqdLEREREZEeIKHD7zH90gAorWly\nuBIRERER6QkSOvymBDwA1DSGHa5ERERERHqChA6/qX4vALVNCr8iIiIi8ukldPjdt/Jb2xhyuBIR\nERER6QkSOvwm+92AVn5FREREpHMkdPj1e9z4PC5qFH5FREREpBMkdPgFSPV7qNUH3kRERESkEyR8\n+E0JeLTtQUREREQ6ReKHX638ioiIiEgnSfjwm+z38PL6El78sNjpUkRERESkm0v48LuxuAaAa5as\ncrgSEREREenuEj78etyxEvunJzlciYiIiIh0dwkffh+59DMMyEhiQ3EN1z3+vtPliIiIiEg3lvDh\nd2SfVC6YPgiAv7+z0+FqRERERKQ7S/jwC9A/4+MtD83hqIOViIiIiEh31i3C79lTBvCFyf0BKK9r\ncrgaEREREemuukX4dbsMZ0zoB0BpjcKviIiIiBydbhF+AXJS/QCU1Sr8ioiIiMjR6TbhNzelJfzW\nNDtciYiIiIh0V90n/Las/O6qanC4EhERERHprrpN+A143UwalMF9/93M65vLnC5HRERERLqhbhN+\nAe6/ZBr9M5L48T/XOl2KiIiIiHRD3Sr8Zqf4OX18P7aX1xOO6Hy/IiIiItIx3Sr8AgzOChKJWnZV\nNjpdioiIiIh0M90u/A7KCgKwY2+9w5WIiIiISHfT7cLv4GyFXxERERE5Ot0u/OalBfB5XGwprXW6\nFBERERHpZrpd+HW7DMf0S+Pl9SVsKqlxuhwRERER6UbaHX6NMW5jzHvGmGfiWVB7jO+fxtayOubf\nsczpUkRERESkG+nIyu93gXXxKqQjhuUkt95uDuuUZyIiIiLSPu0Kv8aYgcAZwB/jW077XDRzCJ8d\nkQ1AUaUudywiIiIi7dPeld87gR8BCbHMGvC6+f4powHYVl7ncDUiIiIi0l14jtTAGPN5oMRau8IY\nM/cw7RYCCwFyc3MpKCjorBoPqqrJAvDSG6sxu71xHUs+ndra2rjPB+leNCdkf5oP0pbmhLTVmXPC\nWGsP38CYXwP/A4SBAJAGPGGtvehQz8nPz7cbNmzolAIPxVrL5J+/yOkT+vHrcyfEdSz5dAoKCpg7\nd67TZUgC0ZyQ/Wk+SFuaE9LWoeaEMWaFtXZaR/o64rYHa+311tqB1tqhwAXAK4cLvl3FGMP4AWl8\nsKvK6VJEREREpJvoduf53d/4/ums311DKJIQW5FFREREJMF1KPxaawustZ+PVzEdNX5AOs2RKOt3\n62IXIiIiInJk3Xrld/rQLAB+V7CJqoaQw9WIiIiISKLr1uE3Lz0AwL/X7uHulzc6XI2IiIiIJLpu\nHX4Brj9tDAA7K+odrkREREREEt0Rz/Ob6C6bM4JXN5Wxp6rR6VJEREREJMF1+5VfgMFZQbbv1cqv\niIiIiBxejwi/Q7KDVNaH2FvX7HQpIiIiIpLAekT4PXZIFsbANUtWOV2KiIiIiCSwHhJ+M7ly3kj+\n+1EpJTXa+ysiIiIiB9cjwi/A6RP7YS28sq7E6VJEREREJEH1mPCb3zeV1ICHD3ZVO12KiIiIiCSo\nHhN+jTEMy0nmlfUlrCmscrocEREREUlAPSb8AgzNTqaosoEz/9+rTpciIiIiIgmoR4XfoM/detta\n62AlIiIiIpKIelT4nT40q/V2ZX3IwUpEREREJBH1qPB77tQB3HzOeACKKhscrkZEREREEk2PCr/G\nGCYNzAAUfkVERETkk3pU+AXon5EEwM699Q5XIiIiIiKJpseF38ygl+G5ybz4YbHTpYiIiIhIgulx\n4dcYwzmTB/DW1r0UVmj1V0REREQ+1uPCL8DZUwYA8NSqXQ5XIiIiIiKJpEeG30FZQaYPzeSpVUVO\nlyIiIiIiCaRHhl+A0yf046PiWraW1TldioiIiIgkiB4bfk8ZlwfAS/rgm4iIiIi06LHhd0BGEkOy\ng6zYXuF0KSIiIiKSIHps+AWYMiiDlTsqsNY6XYqIiIiIJIAeHX6PHZpFSU0Tv3lpI9WNIafLERER\nERGH9ejw+8WpA5k5PIu7Xt7IH5dvdbocEREREXFYjw6/ST43j1w6k9F9U3h9U5nT5YiIiIiIw3p0\n+IXYFd9OHNOXVTsrqW8OO12OiIiIiDiox4dfgKmDMwhHLRv21DhdioiIiIg4qFeE3/y8VAA+Klb4\nFREREenNekX4HZQZJOB18VFxrdOliIiIiIiDekX4dbkMo/qkatuDiIiISC/XK8IvwMSB6azeWUkk\nqgteiIiIiPRWvSb8zhiWRU1TmNE//jdltU1OlyMiIiIiDuhV4RcgErUsXrbF4WpERERExAm9Jvz2\nS0/iH5cfx+zRuTz5XhFRbX8QERER6XV6TfgFmD40iy9M6k9pTRMf7q52uhwRERER6WK9KvwCzMnP\nxes23Fuw2elSRERERKSLHTH8GmMCxpi3jTGrjTEfGGNu6orC4iUnxc9ls0fw7JrdFFc3Ol2OiIiI\niHSh9qz8NgEnWmsnAZOBzxljZsa3rPiaODAdQOFXREREpJfxHKmBtdYC+y6N5m356tafFstJ9QPo\nlGciIiIivYyJZdsjNDLGDawARgL3WGuvPUibhcBCgNzc3GOXLFnSyaV2ntL6KD9c1sDXxvuYM9Dr\ndDk9Xm1tLSkpKU6XIQlEc0L2p/kgbWlOSFuHmhPz5s1bYa2d1pG+jrjyC2CtjQCTjTEZwJPGmPHW\n2rVt2iwGFgPk5+fbuXPndqSOLtUYivDDZc+TM2AYc+eOdLqcHq+goIBEng/S9TQnZH+aD9KW5oS0\n1ZlzokNne7DWVgJLgc91yugOCXjdpPo9lNZo24OIiIhIb9Kesz3ktqz4YoxJAk4G1se7sHhL8rn5\ny+vbqKxvdroUEREREeki7Vn57QcsNca8D7wDvGitfSa+ZcXfqL6xfSOTf/4it7/Q7bO8iIiIiLTD\nEcOvtfZ9a+0Ua+1Ea+14a+3Pu6KweFu0YBLpSbEPu92zVBe8EBEREekNet0V3vbpl57Eg1+f0fpz\nXVPYwWpEREREpCv02vALMGlQBov/51gAPiqucbgaEREREYm3Xh1+AfLzUgFYu6va4UpEREREJN56\nffgdnBVkUFYSL31Y7HQpIiIiIhJnvT78GmM4bXw//vtRKX9cvsXpckREREQkjnp9+AW4Yt5Ihuck\n8/tlW2jP5Z5FREREpHtS+AXSk7xcNmc4pTVNbC6tdbocEREREYkThd8Wxw3PAeDNLXsdrkRERERE\n4kXht8WgrCRSAx7W79FZH0RERER6KoXfFsYYjslLY/1une9XREREpKdS+N3PmH6pvLu9gufX7nG6\nFBERERGJA4Xf/Zw4pg8A9yzd5HAlIiIiIhIPCr/7mZvfh0uOG8LWsjpKqhvZW9fsdEkiIiIi0okU\nftsYnptCbVOYU+5cxrWPv+90OSIiIiLSiRR+2xiRmwJAZX2IrWV1DlcjIiIiIp3J43QBiSY/LxWf\n20VakofdlQ1YazHGOF2WiIiIiHQCrfy2kZvq5+0bTuLyOSOoa45Q3RB2uiQRERER6SQKvweREfTR\nPyMJgIV/fRdrrcMViYiIiEhnUPg9hH7pAQDe2rqXwooGh6sRERERkc6g8HsIo/umtt7eVFLrYCUi\nIiIi0lkUfg8h2e9h9U9PAeCjYl3yWERERKQnUPg9jPSglz6pfjbsUfgVERER6QkUfo/gsyOyef6D\nPbram4iIiEgPoPB7BN+eN5L65ghPvlfkdCkiIiIi8ikp/B7B6L6pjMhNpmBDCVUNIafLEREREZFP\nQeG3HU4c04flG8uYdNN/eOndCFHeAAAgAElEQVTDYqfLEREREZGjpPDbDlfMG8m5UwYAsGxjqcPV\niIiIiMjR8jhdQHeQEfRxx/mT2VPdyModFU6XIyIiIiJHSSu/HTB1cCbrdtdQ1xTW/l8RERGRbkjh\ntwOOHZJJJGqZfdtSJt30H8KRqNMliYiIiEgHKPx2wJTBGQCUt5zzt6xW5/4VERER6U4UfjsgI+hj\ndN+U1p93VzU4WI2IiIiIdJTCbwc9uvA4Hv/WcQAUVzc6XI2IiIiIdITO9tBBmck+hhFb/d1dpfAr\nIiIi0p1o5fcoZAa9uF2Gl9YVs3NvvdPliIiIiEg7KfweBWMMeWkBXttUzufuXEZzWGd9EBEREekO\nFH6P0t1fmcLnxuVR1xxhU0mt0+WIiIiISDso/B6lqYMz+cGp+QCsKaqkrLbJ4YpERERE5EiOGH6N\nMYOMMUuNMR8aYz4wxny3KwrrDoblJBPwurj28TVM++VLNIUjTpckIiIiIofRnpXfMPB9a+1YYCZw\nhTFmbHzL6h7cLsNxw7Nbf15bVOVgNSIiIiJyJEcMv9ba3dbalS23a4B1wIB4F9Zd/PKcCUxtufLb\nO9sqHK5GRERERA6nQ3t+jTFDgSnAW/EopjsakJHEE9+eRX7fVP69ZjeRqGVrWZ3TZYmIiIjIQRhr\nbfsaGpMC/Be42Vr7xEEeXwgsBMjNzT12yZIlnVlnwlu6I8QDHzZjAAtcPdXP5D66hghAbW0tKSkp\nR24ovYbmhOxP80Ha0pyQtg41J+bNm7fCWjutI321K/waY7zAM8AL1to7jtQ+Pz/fbtiwoSN1dHv1\nzWHG/vSF1p/PnTqAO7402cGKEkdBQQFz5851ugxJIJoTsj/NB2lLc0LaOtScMMZ0OPy252wPBvgT\nsK49wbe3Cvo8DMhIAuCzI7JZur6E9q6qi4iIiEjXaM+e31nA/wAnGmNWtXydHue6uqVHLp3JPV+Z\nSmbGTirqQ3hvymTonUN5aM1DTpcmIiIiIsARN6Vaa18FTBfU0u0Nzg6yfNeTPPzhvQTNxbijA9hZ\nuZ2FTy8E4MIJFzpcoYiIiEjvpiu8dbIbXr6BmuhWin0/IWJKSAufR1r1Ldzwwm+dLk1ERESk11P4\n7WQ7qnYQMcVkhS7DawdS6X2QPYHvs6umlFV7VjldnoiIiEivpvDbyQanDwYTwR8dTXr4Qvo23kJK\nZD5hVxFTfj+F2X+ezWMfPkY4Gna6VBEREZFeR+G3k9180s0EvUGKAt+g2Hctxf4fkxqdx4ykP7Po\n5EUUVhey4B8LGPbbYdzy6i2U1Zc5XbKIiIhIr6Hw28kunHAhi89czOCMfhhjGZIxgBNHD2dvdRaX\nH/tdNl65kacueIr87Hyuf/l6Bv1mEN946hvaEiEiIiLSBRR+4+DCCRey7eptRH8WZdvV2/jOCXNp\nDkf5f0s3saaohrPyz+Kli19izbfWcMmkS3hk7SNM+f0U5vxlDo9/+Li2RIiIiIjEicJvF5g+NIu+\naX7uLdjMV/7wJo2hCADj+4znvs/fR9E1RSw6eRE7qnbwxX98keG/Hc6tr95KeX25w5WLiIiI9CwK\nv13A7TI8feXx3HLuBOqbI7yx+cBQm5mUyfc/+302XbmJf57/T0Zlj+K6l69j4G8G8s1/fZP3i993\nqHIRERGRnkXht4v0SQ1w9pQBBH1uvvPwStbtrv5EG7fLzRfGfIGXL36ZNd9aw8UTL+bhNQ8z6b5J\nzP3LXJ5Y94S2RIiIiIh8Cgq/XSjgdfOXr80gqSUA1zcfOsiO7zOe35/5ewqvKeT2k29nW+U2zlty\nHiPuGsFtr93G3oa9XVi5iIiISM+g8NvFZgzL4q4LprClrI5fPLMOay1FlQ2HbJ+VlMUPPvsDNl+1\nmSfPf5IRmSO49qVrGXjHQC7916XaEiEiIiLSAQq/DvjsyBwuPWE4j7y9g+NvXcqsW17hzS2H/3Cb\n2+Xm7DFn88olr/D+5e9z0cSLeGjNQ0y6bxLzHpjHk+ueJBKNdNErEBEREemeFH4dcuWJIwFaV32f\nWFnY7udO6DuBxWcuZuf3dnLr/FvZUrGFc5ecy4i7RnD7a7drS4SIiIjIISj8OiQ14OW44dkAnDY+\nj3+v3UNTuGMrt9nBbH4060dsvmozT3zpCYZlDuNHL/2IgXcM5LKnL2Ntydp4lC4iIiLSbSn8OugP\nl0zjjetP5Pzpg6hpDJP/4+e5t2AzS97ZSSgSbXc/HpeHc445h6WXLGX15au5cMKFPPj+g0y4dwIn\nPnAi/1z/T22JEBEREUHh11Epfg/90pOYNTKHYTnJANz6/Hp+9Pj7LF62hf9+VEq4AyEYYGLfifzh\nrD9Q+L1Cbp1/K5srNnPOo+cw8u6RLHp9ERUNFfF4KSIiIiLdgsJvAvC6Xbx0zRy2/vp0nv7O8Uwa\nlMHtL2zgkvvf5jsPv3dUfe6/JeLxLz3OkPQh/PDFHzLwNwO5/JnL+aDkg05+FSIiIiKJT+E3Qbhd\nBmMMEwamc/cFU1rvf/6DPazbXY21lurGUIf79bg8nHvMuRR8tYBVl63iy+O/zAOrH2D8veOZ/+B8\nnlr/lLZEiIiISK+h8JuABmcHefP6k3j3x/NJ8rr53qOr+OwtrzD15y+yqaTmqPudlDeJP571Rwq/\nV8gtJ93CR+UfcfajZzPy7pH83+v/py0RIiIi0uMp/CaovPQAOSl+fv6FcazfU8PuqkbCUcv8O5bx\n6Ds7KNhQwvNrdxON2g73nR3M5trjr2XLd7fw2ILHGJw+mB+8+AMG/mYg33rmW3xY+mEcXpGIiIiI\n8zxOFyCHt2DaII7pl0YoEuX//vMRr24q49rH17Q+ftNZ4zh9Qj9yU/0d7tvj8nDe2PM4b+x5rNqz\nirvfupu/rP4L9624j/nD53PVjKs4fdTpuF3uznxJIiIiIo5R+O0Gxg9IB+C3F0ymujHM39/eQWFl\nA6XVTfzsXx/ws399wICMJK6eP4oF0wYd1RiT8ybzpy/8iVtPvpU/rvwj97xzD2f9/SyGZQzjOzO+\nw9enfJ2MQEZnviwRERGRLqdtD91IdoqfYTnJXH/6Mdzzlanc/ZUp5KTEVnyLKhv44WPv86vn1vHa\npjI2ldRgre3wtoicYA7XHX8dW7+7lX8s+AcD0wby/f98nwF3DODbz36bdaXr4vHSRERERLqEVn67\nsb5pAd654SS2lNVR3RDib2/uYPGyLSxetqW1zczhWWSn+OmXFuDHnx/b7r49Lg9fHPtFvjj2i7y3\n+z3ufvtu7n/vfu59915OHn4yV864UlsiREREpNtR+O3mjDGMyE0BYNLADKYPzSQ7xc+Db2xj+cYy\n3tyyt7XtVfNHkRbwUlHXzP2vbeWCGYMZkJF0xDGm9JvC/V+4n1vnH7glYnjmcL4z/Tt8bcrXtCVC\nREREugVte+hBXC7DBTMGc/LYvvz+f47l3gunMmNoFhcfNwSAyTf9h7PveY1jf/kid7+yiUUvbCAc\nibJiewWzb1vK06t3Ud8cPmT/ucm5XH/C9Wz97laWfHEJ/VP7c81/rmHgHQO54tkrtCVCREREEp7C\nbw8V9Hk4bUI/llx+HDedNY4ffS6fqIVVOyv5+qxhjMlL5cn3ijj+1qVc+uC77Nhbz5WPvMfYn77A\nqb9ZxuubymgMHfziF163lwXjFvDMBa/w+tffZsG4BfzpvT8x9ndjOfVvp/LsR88StVEeWvMQQ+8c\nyordKxh651AeWvNQF78LIiIiIgfStodewBjDt+eO5KQxfbFYxuSlsXNvPQ++sY03tpSzuaSO608b\nw6//vR6ADcU1fOWPb+HzuHjnhvn8/r+bCXjdnDouj5wUH9kpfqy1nLiogKxkH98/5dd8ffyPWb7r\nUX73zu/4/COfp09yH6rq3TRHa6EvbK/azsKnFwJw4YQLHXw3REREpDdT+O1F8vNSW28Pygpywxlj\niUYtdc1hUgNeLpszgjWFVTSGIyy47w2aw1Em3fSf1ufc8eJHeN2G40bkMK5/GuV1zZTXNXP531YA\ncOKYk5mbPpem7Dd4eus9NJmtGAIs2vQonmg/6puL+dYz32Jt8VoCngABTwC/x//xbXfsttv4SPEn\nfeL+tu09ru4xfR9a8xA3vHwDO6p2MDh9MDefdLMOAERERBxirO34FcKOJD8/327YsKHT+5WuU9MY\n4qI/vc3qnZUcNzybr3xmML9+bh27qhoPaDdxYDrvF1YBMDgryI699QCEzC7q3AWETTFNrnVYQljT\nSJRGvG5DKNr8qWt0GRd+d4Akbywg+z0BkjwBvC4fQV8sPLvx4nb5yUxKjgVnd4DqBshLSyXVHzxs\nuN7//uawi/RAkLRA8gH3+9w+jDGfqC0UieI2hkc+eJiFTy+kPlTf+ljQG2TxmYt7ZQDedyBwZd8r\nubv47l57IKADohjNh49pTsRoTsRoPnzsSHPCGLPCWjutI312j6Uz6XKpAS+PLpyJz+3C5YqFuxNG\n5XDT0x/ytVlDeXbNbkJhy0/PHEtTOEJjKEpawMOG4hre31nFj169gqrq7XiiAzk382Ze3+PFRSqG\n2KnRLFGSfBaXCVHT1IA1ISzNeNxhXK4QDeGm1p+NCdMYaSQ7xeD3RtlZWYWlGQhhCZEbcFNZX0+1\nbcKbYtheWU2/DBeldXtpCjcRpZlkf5T0oKG6sZ6qxjosYTDNWD79wZ/X5SfoDeB1+/EYH6n+IEUV\nIZK9SVRHttIUacDYQKxmE6U+VM/Cpxfy3MbncBnXx1+4qG2KkOL34nW7cRkXBtP6eNTGtrA0hwFr\nyAj6D3h835cxhoq6EJnJfnxuDwZDc9gS9Hlb+3l3WwWTB2WR4vfSGIqS4vdR0xgm4PWS7PNizIH9\nGkzrfW7jxhjDyu2VDMpMZkBm8iHrCEcs9c2WnJQAL255kVtfvZWmSBM70nawvWo733jqG2zeu5nT\nR50OgCE21/YdUCTKz82RKLWNYfbWNTO6b9oh27enzyfWPcEP/vMDGsINQGxL0KX/upSaphoWjF1w\nwPO66rYTHlrz0McHhr18a9QB7wW9973QnIjRfPhYvOaEVn4lLvafsItGL+IHG35E0JvKL+fcywD/\n8UwdksGQ7GTcxrClrI5tZXU0hiN8Zlg2oUiUbeV1vLapjNKaJpL9HpZvLGNTSS0A504ZwBPvFbWO\nlZXsIz3Jy9ayuk/UccaEfvRLD/DHV7eSnuSlqiEEgM/joikcAcIEfFHy8/zsrq5hV1UNluaWlerY\n9yRfhLrmRmgJ6LYldO973BJiYJaHXVXVhKLNsTatjzUTNfUYIEoDUVONJYLLJhH0BXEZi9tlsERp\nDodpDMfOthHwGlwGGkIhvG5D1EYJRSItYT325XbZlp+jRK3FEo3vH6r0aPuH9EPdxhqiWDzGhcVg\nDLgO037/29YaItEoPo8bg6GisYKojc3ZZHcydZHY31+XcZETzDlojdbC/nk9GrWxg3PLYYN8JGpj\ndR4h6+/ro+04WOAwz4229I/5+H0MRaJYG/u35pA1uWKtS+pKiNhPfsDYbdwMSh9EJAo+t7v1AHT/\ng9F9B5773u99tw/WLhwFf8v7f7h27enPWvC42tSEweWK3cYa3C7XQfuNRsHb5vVErOWBVX+murka\ngHmZ81hasRSANH8a35r2rcP/2R3uD6ht2w4c9DnR791v301VU9Un7k/3p/O9md9rHWvfn0fbv3P7\nf983Vnvad6SPePXbtv2lT19KaX0pAD8b/jNu2nITAEPSh7Dt6m372mrlVxLDviOyG16+AYAhGYNa\nflXx5U+0HdknhZF9Ug64r39GEp8d8fF/gNZa/rB8CwGvmws/M4TbF0ziV8+t45wpA1ov//x+YSUP\nvL6dH5w6mvN//yZZyT7uuXAq1lpW7axkT3Vja/h96/qTuPHpD5g9KpeCj0opq2liUr++XDknh1c3\nljG6byp/e2s7lfUhJuWkc/M5ExiTl8rWsjqufnQVP/38WPqkBVi3u5rHVxTy8voS0tu8rqtOHMld\nrz9CY3MQb3QQ1tTjsXmxB00jff3pFFc3tbYfnBWkqL6BSNRimmL/AR/KgIwkiipjK4d+j4umcCxE\n7AvDxsDgzADb99btF4qjLS32Bego+wfpcDQKxra0+fg5v71gErf8ex27qxpa7o89d0h2EtvK6w7s\ny3wcyPf1neJ3U9MUar2v0fU+2e58akNeoqYSrAsXqS3t92c/8d3ncRG1lnBk3zix+zOCXirrQ3hc\n8N35o/j32j1UNTQzMDPI+4UVNIQiuF1wytg8XlpXHAsmWNICXqobQ4cY7+B1nDmpH1lBH395YxsA\n/dIDZCR5GJKdTF66n7+9uSP2XrY85/iROby6qfSAHqOEcOHZ788DIqaScX0H0hCKMLJPCs2RCNvL\n6qmobyI1yUN5TRNNkQhBn5u65gh+z74DpEjrWPu+D8kO4nO7GJSVRGVDM0UVDTSEwq3zHyypfjc1\nTeEDnut1GyyWUCT2eFqSh+wUH8bA2qIqItGPD7728XkN1sLQ7CCbSms4ZWxftpbVUVRZzylj+5KW\n5GFLaS3LNpZhsKQm+whFonjDGwAvLptCqkkmFNneWsvo1GnUNocxQH5eGqt3VrKzIrYKlpHkpV96\nEg2hCNvK6xiWk8yOvXUMy0lhUGYSb2/di8dtaGiOkBH0EbGW8trY37OcFD/90gNsKavD53Yxrn8a\nbpeLHXvr8HtcDMkO0hyOsuyjMpL9bsb2TycUifLejkqCfjcn5vdh/Z5qyuuayUnxU1nfzKDMIO9u\nr6B/RhJeNxRWNDAiN4WPimsA+PzUAby0roRo1JKd4ic96GVLaS31jbH3fnTfVEqq/4vLxH7b5YsO\nJ2KqCbsKidgI47I/Q8GGEsb0S2FEbjKFFfVYopTUNBKKRBmWk8Teumbc7th7V1LTiN9jyEnxUV7X\nSDgSZUBmEo3hMB/uqiI14CbF78brcVHb1Exm0EtxTQNBr5ucVB9uF4SjURpDYQJeV8sBiiVqLZFo\nhIi1GCzbymvxuk3rc+qaQpTVNhH0uXC7Y1vnvG5DKBIhxe/B54n1W9sUIhyJEvS7iUYjhK3FZSyN\noQiWELGjDMuyiuUt08xQ3VjNnW/eGfu7YyFqLZ6W30ju/9u7ff9mtj1AilhwG4PLFfu/5GCituUA\npmUMay2ufcctNnafa7/jmP3HsjZWR3uj75F+47jvoLC1WUvHVU1V3PjfG9s5Ss+zs3Fn6+0dVTs+\nVV9a+ZW4KygoYO7cuV06Zl1TmIiNhRuA5nAUt8vw5pZyAl4Xxw7JOmIfFXXNrCmqYvbo3CO23bCn\nhiSvmySfm91VDRRWNHD6hH6f+PWVJ9qXJE8qv/38jXx18lf421s7aA5HyUjycu7UARhjqG8OE7Ww\nsbiG9CQvf35tG8ePymHWyBweenM7Jx3Tl6xkHy+vK+aj4hr+sHwrADOGZnHq+DzmjM5leE4yTeEo\ni5dt4QuT+zMgM4mtZXUs31jGB0VVnDmpP6+sL+Gvb27ntxdM5owJ/XC7DA+9tYO3t+4lM+jlgTe2\nt76+JK+bu788hW8++C4At31xIudMGcC72yrok+bnhifXtF5QZfVPT+H5D3bjdrmIRi3nTB3ANUtW\n8/SatUSjPlykEPSEKeIh/NFjCLpz+OZnTqKmMcy28jp+cfZ4tpfXk+x3s7m0jpc+LGbBtIEMyUrm\nkXd2EPC4OW5ENi9+uIfSmiZG9U1tvaqhy8T+k2rrJ58fy8NvbWdzaR3ZyT7K6z7ecz4vP5e89ACP\nvL2T+cf04YLpg2mORBnVJwWP28VVj7xHks/NzGFZvLy+hA92Vbc+97I5w/nj8q0tofBjeWkBLp09\nnF888+EB93tchrB7K4SGtt7XbLYQNU0EXKkQHth6vzGxC9cMz02mtKYJlzGcOKYPL3ywB4DXN5e3\nts1N9XPB9EFMHZzJvQWb2V3dQHM4SnF1Ez63i4kD0/lwdzX1zQeuLp4wKofjRmRz2/MbmDUym9c3\nlx/yoMvrNlw9fzTHDsnkgsVvHrwRkNwSztuaOjiDoTnJPL92D3npAbaUfvI3Ne0R9Lk/8Tra2n8e\nDM9N5syJ/fntyxtbHx+ek0xtU5iSmlgoTgt4aAxFaY4c/Lcnx/RLY0tpbetB5qG4XYaRuSlsaAm+\nn0aY8tiHeqNtD6tjUvwerLUHfa+PpO3fkySvm0jUEvDGtrlV1odaH0vxe6ht+vj876l+D+cdO5C/\nvL4tNp+jlhnDsnh7694DfrvW1jNXHs+lD77L7jafG2lfwVX4XJk0t3n/h+cmE/C4KalpYnhOMm9v\ni/0bNLZfGkGf+6BzPjPoxe1y4XUbJg/K4N9r9xzwWn9x9ji+9+hqADKCXuqbIq3z4ltzR/DtuSP4\n82vbuOPFj9pVem6qn2HZydxx/iQCXjfn/u516pvD/OzMcTzw+jbe3V5BVrKPP1w8jfPufR2f20Vz\nxq8or04lI3QRbtKpc72G1+bhdaWQFxzMnpoGsoJe9tY3A5Zx/VNxu1wMyw7i97oYnpvMvQUbqWwI\nEfC6Wk5baslO9jK2fxq7KhvAWJK8bpL9bjwuQ2M4wrj+aTzw+jbaHuDuWzBJT/Lw4Dem8+Dr2ymu\naeRnZ47l9hfWsaaoil1VDcwYmsUPTh1NY3OYR9/ZySsbiknxe+iT5qeqoZnClgPY/RcuwJIW8HD7\ngokMzgry3Jrd/L+lG/G5DaeOz+OpD18m2jyQSu9f+cnoi/jJ5p8An37lV+FX4s6J8JtI4vnBhcZQ\nhN+8+BF56QG+NmtYh59f2xQmxX/wXwCt2lnJ2fe8xrlTBvDzs8eT4vdQWFHPjvJ6Pjsy5xP9jP/Z\nC+Sk+Hj3xyd/oq9o1PLQmoe54l8/wzaP4EejTuOnW7/XaR/+e37tHoI+NwMyk3hn614GZgaZMDCd\nKx5ayeDsIL86ZwLv7ajgHysKuXz2CGbfvpTMoJcnvz2LoTnJAJTXNpGd4j/sOCXVjVz0p7f4qDi2\nBWfbLWfw9OpdrN9TzT1LNwPw0jWz6ZsWIDXg5fYX1rfeD3DNyaPJ7vM2Vzx5D0kN51Pi/zFRU9v6\nPkzLPYu8tABVDSHSk7ykB72HrOX9wkq8bhej+qTgMqZ1b/4+jaEIf397B6eMy6N/RlLrb0DG5KXx\n+MpCzpzYv7X/LaW1DMtJZldVI+/vrOR/n1xDRX2IE0blkOL3MCYvjS9OG9h6Rci3tpSzY289g7OC\nbC2rY3ReKh8UVXHciGxu/NeHvLGlnEULJrJzbwN3v7KR0yf04/8WTMLjdmGtxRjDDc8t4cHlNbhb\nfhtS5L8cv6+J2+bfybljzsXncfHmlnL++V4Rl80ZTpLXw+DsIF634f3CKv60fCvZKT5SA16+c+JI\nNpXUsuTdnZwzZQDThmTywa5qxuSl4nHHluuqG0NMvDF29pqPfnkakajlra3l1DVFOHFMH0LRKDc/\ns45QJMpVJ43i0Xd38pfXtjEwM4m/fuMzLN9YyhMrixjVN4UH39iOz+PiqStm8ZN//v/27j04q/rO\n4/j7mxu5EUJuQG7cLwtIABWkXoZLRdwt0HYcVm23blfXztYdtdZRrLPr7HTcmd126a7rbrcdq3VX\nl0rRVkq7tlRlsFVREQjIRSAQDBAScg8hlyfPb/84J48PIUDCPE+InM9rhknO75yc50fON9/n+zvn\nd86zm9KcdA7UtPLkl2Yyqzibl7dV8ejL5YTCjpTEBJb8SQH/sqqMlvYQyYkJ7Kxq5KqiEaSnJPKr\nncd5s+I91u/5OZkdX6Mh6XkA0qyUsRk3MauwkAUTctlf3cLG8uP8zcKJdIbCfHluMaGw43d7qrll\nxmg27jxB/vBhLC8bgwPePniKSQWZZA5L5u1Dp3ho3c5IbOz8+6U0t3fREQrzi+1VLJ5WQIIZj75c\nHhkQLJsxmkO1rRxvbOdY4xlmFGZxsObTAUB2ejI//otrWPWjdxiemsTorFQeunkK10/O4+k3DpJg\nxrbKet4/0nBWXP7XV+eSlzmMyro2po0Zzub9tTzzVgX33DiBF977iIOnX8VCheQlXEtduJzu5HLm\nF9zJvNIp1LZ0UJSdxvTCLB742Y7IPlddU8wHRxqoOHWaz03MZevhejKHJUUK8aLsNJaXFeKco7al\ng45QmF/vOgF4V27mjh3J1op6TvlXCFISE1g6YxQby71tZpdkk5eZwu/31vT5tzghP4Omtq7IoHrV\nNcUsnFrA/Wu3E/JHGQsm5NIe6mbviWbG5mQMeIDkCDO+IExt47DIgKc/A0Hwrio+dPMUHnxpR5/r\nx+dl0BkKR64m3jApj8mjMmk600VtSwdvHTjV58+NzkqlutkbzPScAc8clsSZru5zTggArCgrpKGt\nkz3Hm5lRNILHbp3GO4fqWLPpYzpC3RQMT430obdOO8y3Z3Xw8McPn/O+oeJXhqSgF7+fZfuqm5k6\nani/5rK9se8kE/MzGZubcd5thsqd3NVN7aQlJ16wuDwf5xwPvrSDxdMKWDm7KNK+taKOyro2Vl1b\nEmk7cuo03924h5EZKazfVsVzf3kti6YVfCbu5K5r9ebbpyYnDujnnHPUn+6MDCQa2zrJSk0+pzgH\neOrttazZkEVx5mmqMh+J++/hpfePUpKTftaUqgvpDIVJSjh3YPHcHw8zf3wu0wuzzvuzzjlCYe/y\nfH/+fl7c9SKPb/pHjjbvpTS775joGThcipPN7ZFjOSLtwnEf/TrtXd1sq2xgwYRcEhKMlvYuXt9b\nQ0lOGlePzWF/dQtFI9P6HESHusOEwt6gq6rhDEfr27h/8aTIgKRHd9i79yHye+hHjmjtCLFhx3FK\nctK4cXI+zjnCzjv7fqi2lZHpKSQnGs+8dZi/vmnCOf0bt/rXALz3+BIKhnsDzv3VLby+9yTzxuew\neFoBlXVtFGankZKUwMGaFj6/ZgsAa1aVMbskm+6w40dbKvjuypkkJhh/+78f8rs9J/nlfdczuySb\nL//nH/nwaCMT8jK8qRT+huYAAAffSURBVDZJCTx1+2yKstNZ/vQfAPjDo4t47JVdTMzP5PZ5Jfzf\nrmpOtXaQk7ubf397LVXtWyjOKuKxGx7nG/PupMWfptVwuovS3HQq607zd69+xOpl06ht7eCj4038\n82v7eeqOOYwZkcqOo43cc+N4zIxtlQ0cqm1lyqjhvHOojuONZ/jWzVPIyUiJHNc5pdn+FJ5Pj1Go\nO8zqV3axflvVWb9DM/jmwomU5qRz29Ul7D7WxCPry5k/IYe7PjeOYw1nuGlKPt/77T7SU5K4b9Gk\nPuPtk/o2Xni3kpPN7bR2hJhTOpKpo4bT2R2mrbOb3x98l9e2p3HHlMOsbf5+TJ72oOJX4k7Fr/QW\ntJjoCHXzxt4als0cfVmfsDAUvbrjGHbyY1bcsuhyd0WGkHjniF1VTVScaj1rAHsx5VWNjM/LYHhq\n34OH5vYu3txXw4qyQsy8IvyFdyv55sJJbNpzkhsm5VGam0532DHxO78BvKtHseSco7yqiVnFI2Ke\nazbsPM62I/V0hMI8sXwGTWe6GD0iNaav0Zd6//MElo5q454vLjlnvW54ExEZgoYlJXLrVWMudzeG\npJWzi9i8+cDFNxSJoauKR3BVcd/zqc9nVnH2BddnpSafVUxPzM/kieUzALhzfmmkPTHBePJLM8lI\niX0JZmaUlVy4n5dqRVkhK8oKI8tpKQO7KnSpcjJSWPeNBWzevDlm+1TxKyIiIjKIvjJ/7OXuQqD1\n/QDCKGb2rJnVmNnuweiQiIiIiEi8XLT4BX4KLItzP0RERERE4u6ixa9zbgtQPwh9ERERERGJq349\n7cHMxgEbnXMzL7DNvcC9APn5+VevW7cuRl2Uz7rW1lYyMzMvvqEEhmJCoikepDfFhPR2vphYtGhR\nfB511p/iN5oedSbRgvZYK7k4xYREUzxIb4oJ6e18MXEpjzrrz5xfEREREZErgopfEREREQmM/jzq\nbC3wDjDVzKrM7O74d0tEREREJPYu+iEXzrk7BqMjIiIiIiLxpmkPIiIiIhIYKn5FREREJDBU/IqI\niIhIYKj4FREREZHA6NeHXAx4p2YtgD7lQnrkAacudydkSFFMSDTFg/SmmJDezhcTY51z+QPZ0UWf\n9nCJ9g/00zbkymVmHygeJJpiQqIpHqQ3xYT0FsuY0LQHEREREQkMFb8iIiIiEhjxKn5/HKf9ymeT\n4kF6U0xINMWD9KaYkN5iFhNxueFNRERERGQo0rQHEREREQmMmBa/ZrbMzPab2UEzWx3LfcvQZGYl\nZvamme0xs4/M7AG/PcfMNpnZAf/rSL/dzOwpP0bKzWzu5f0fSLyYWaKZbTezjf7yeDPb6h/7l8ws\nxW8f5i8f9NePu5z9lvgws2wzW29m+8xsr5ktUJ4ILjP7lv+esdvM1ppZqnJEsJjZs2ZWY2a7o9oG\nnBPM7C5/+wNmdld/Xjtmxa+ZJQL/AdwKTAfuMLPpsdq/DFkh4NvOuenAdcB9/nFfDbzunJsMvO4v\ngxcfk/1/9wI/HPwuyyB5ANgbtfxPwA+cc5OABuBuv/1uoMFv/4G/nVx5/g14zTk3DSjDiw3liQAy\nsyLgfuAa59xMIBG4HeWIoPkpsKxX24BygpnlAE8A84F5wBM9BfOFxPLM7zzgoHOuwjnXCfwMWBnD\n/csQ5Jw74Zz70P++Be8NrQjv2D/vb/Y88EX/+5XAfzvPu0C2mY0Z5G5LnJlZMfBnwDP+sgGLgfX+\nJr1joidW1gNL/O3lCmFmI4CbgJ8AOOc6nXONKE8EWRKQZmZJQDpwAuWIQHHObQHqezUPNCfcAmxy\nztU75xqATZxbUJ8jlsVvEfBJ1HKV3yYB4V+KmgNsBUY55074q6qBUf73ipNg+FfgESDsL+cCjc65\nkL8cfdwjMeGvb/K3lyvHeKAWeM6fCvOMmWWgPBFIzrljwPeBo3hFbxOwDeUIGXhOuKRcoRveJCbM\nLBN4GXjQOdccvc55jxTRY0UCwsy+ANQ457Zd7r7IkJEEzAV+6JybA5zm08uZgPJEkPiXpVfiDYoK\ngQz6cbZOgiWeOSGWxe8xoCRqudhvkyucmSXjFb4vOude8ZtP9lym9L/W+O2Kkyvf9cAKMzuCN/1p\nMd58z2z/EiecfdwjMeGvHwHUDWaHJe6qgCrn3FZ/eT1eMaw8EUyfBw4752qdc13AK3h5QzlCBpoT\nLilXxLL4fR+Y7N+tmYI3eX1DDPcvQ5A/7+onwF7n3JqoVRuAnrsu7wJejWr/mn/n5nVAU9QlDrkC\nOOcec84VO+fG4eWBN5xzXwHeBG7zN+sdEz2xcpu/vc4AXkGcc9XAJ2Y21W9aAuxBeSKojgLXmVm6\n/x7SEw/KETLQnPBbYKmZjfSvKCz12y4oph9yYWZ/ijfXLxF41jn3ZMx2LkOSmd0AvAXs4tP5nd/B\nm/e7DigFKoFVzrl6P9E9jXeJqw34unPug0HvuAwKM1sIPOyc+4KZTcA7E5wDbAe+6pzrMLNU4H/w\n5ovXA7c75youV58lPsxsNt4NkClABfB1vBMwyhMBZGb/APw53hODtgP34M3VVI4ICDNbCywE8oCT\neE9t+CUDzAlm9ld4dQfAk8655y762ho8iYiIiEhQ6IY3EREREQkMFb8iIiIiEhgqfkVEREQkMFT8\nioiIiEhgqPgVERERkcBQ8SsiIiIigaHiV0REREQCQ8WviIiIiATG/wN18ARRZpHYNQAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#retrieve train and validation summary object and read the loss data into ndarray's. \n",
    "train_loss = np.array(wide_n_deep.get_train_summary(\"Loss\"))\n",
    "val_loss = np.array(wide_n_deep.get_validation_summary(\"Loss\"))\n",
    "#plot the train and validation curves\n",
    "# each event data is a tuple in form of (iteration_count, value, timestamp)\n",
    "plt.figure(figsize = (12,6))\n",
    "plt.plot(train_loss[:,0],train_loss[:,1],label='train loss')\n",
    "plt.plot(val_loss[:,0],val_loss[:,1],label='val loss',color='green')\n",
    "plt.scatter(val_loss[:,0],val_loss[:,1],color='green')\n",
    "plt.legend();\n",
    "plt.xlim(0,train_loss.shape[0]+10)\n",
    "plt.grid(True)\n",
    "plt.title(\"loss\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "plot accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAF1CAYAAADIhX0mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHuFJREFUeJzt3X20ZXV5H/DvwyCSwBgUJ5gwE8FK\npsE3ioC28WUwakAjJNEmYGs0bRaxcVa1rbFk6aKUJKuRtKarCcZFW/NmDNCo7TSONdH21pKqvFhQ\nQakDkjBoVBDRwRJefPrHOZMcL/fOXPid4d4ZPp+1zpqz9/7tfZ7zsOfwnX1+55zq7gAAAA/NQatd\nAAAA7M8EagAAGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAADBCoAfagqm6uqhfO6ViHVNUfTo/Z\nVbVlHscFYHUJ1AAPr8uT/P0kf7HahexNVa1b7RoA9gcCNcAyqur3knxfkv9aVbuq6k3T9WdU1XVV\n9bWqWqiqH5jZ5+aq+oWqur6q7qiq36qqQ5Oku+/p7n/b3ZcnuX8Fj//TVfWZqvpGVd1UVT+7aPuZ\nVXVNVX29qm6sqtOm6x83fdwvTGv4z9P1r6mqyxcdo6vqydP7v11Vv1lV26vqriSnVtVLq+r/TB/j\nlqo6f9H+z6mq/z3txS3Txzi5qr40G8ir6ser6toH0X6A/YZADbCM7n5Vkj9P8rLuPry7L6yq70/y\nB0nekGRDku2ZBO5DZnb9e0l+OMnfSPL9Sd7yEEv4cpIfSfKYJD+d5Neq6sQkqapTkvxukp9PckSS\n5yW5ebrf7yX5ziRPSfLdSX7tQTzmK5P8cpL1mVxNvyvJT00f46VJ/lFV/ei0hicm+UCSX8+kFyck\nuaa7r0xye5IXzxz3VdN6AQ44AjXAg/OTSd7f3X/S3fcm+ddJviPJ35kZ8xvdfUt3fzWTcHr2Q3mg\n7n5/d9/YE/8zyR8nee508z9M8s5pHd/q7lu7+7NV9T1JTk/y2u6+o7vvne67Uv+lu/90esy7u3uh\nuz81Xf5kJv+YeP507CuTfKi7/2D6OLd39zXTbb+TydSWVNXjMvkHxrsfSh8A1jqBGuDB+d4kf7Z7\nobu/leSWJEfPjLll5v6fTfd50Krq9Kr6WFV9taq+luQlSR4/3bwpyY1L7LYpyVe7+46H8pj59tpT\nVc+qqv9RVV+pqjuTvHYFNSTJu5K8rKoOS/ITSf5Xd3/xIdYEsKYJ1AB71ouWv5DkibsXqqoyCZa3\nzozZNHP/+6b7PChV9egk78nkCvhR3X1EJtNLajrklkymlCx2S5LHVdURS2y7K5OpILsf4wlLjFn8\nfN+dZFuSTd39XUnesYIa0t23Jvlokh/PZLrH7y01DuBAIFAD7NmXkjxpZvmyJC+tqh+qqkcl+WdJ\n/jLJ/54Z87qq2jid6vDmJJfu3lBVj979IcUkh1TVodNQvtghSR6d5CtJ7quq0/Ptc5L/Y5KfntZx\nUFUdXVV/c3oV+ANJ3l5Vj62qR1XV86b7XJvkKVV1wrSG81fw/NdncsX77um87VfObPv9JC+sqp+o\nqoOr6siqOmFm++8meVOSpyV57woeC2C/JFAD7Nm/SvKW6bdYvLG7b8hkbvCvJ7ktycsy+dDiPTP7\nvDuT+c43ZTIl4pdmtt2Q5P9lMkXkg9P7T8wi3f2NJP84kwB/RyZBdtvM9isy/aBikjuT/M+Z47wq\nyb1JPpvJBxvfMN3n/ya5IMmHknwukw8d7s3PJbmgqr6R5LxpPbtr+PNMpqH8syRfTXJNkmfM7Pu+\naU3v6+5vruCxAPZL1b343T0AHqqqujnJz3T3h1a7lrWgqm5M8rP6ARzIXKEGYJ+oqpdnMif7v692\nLQD70sGrXQAAB56qWkhyfJJXTb8JBeCAZcoHAAAMMOUDAAAGCNQAADBgzc2hPuKII/rJT37yapdx\nwLjrrrty2GGHrXYZBwz9nB+9nC/9nC/9nC/9nB+9nK+rr776tu7eMHqcNReojzrqqFx11VWrXcYB\nY2FhIVu2bFntMg4Y+jk/ejlf+jlf+jlf+jk/ejlfVfVn8ziOKR8AADBAoAYAgAECNQAADFhzc6gB\nAHh43Hvvvdm5c2fuvvvu1S5lnzr00EOzcePGPOpRj9onxxeoAQAeoXbu3Jn169fnmGOOSVWtdjn7\nRHfn9ttvz86dO3Psscfuk8cw5QMA4BHq7rvvzpFHHnnAhukkqaoceeSR+/QqvEANAPAIdiCH6d32\n9XMUqAEAWDVf+9rX8va3v/0h7//mN785mzZtyuGHHz7Hqh4cgRoAgFUzGqhf9rKX5YorrphjRQ+e\nDyUCALBqzj333Nx444054YQT8qIXvShJ8oEPfCBVlbe85S35yZ/8ySwsLOS8887L+vXrs2PHjpx6\n6ql5+9vfnoMOOijPfvazV/kZCNQAACT5l//1ulz/ha/P9ZjHf+9j8i9e9pQ9jvmVX/mVfPrTn841\n11yT97znPXnHO96Ra6+9NrfddltOPvnkPO95z0uSXHHFFbn++uvzxCc+Maeddlre+9735hWveMVc\n632oTPkAAGBNuPzyy3P22Wdn3bp1Oeqoo/L85z8/V155ZZLklFNOyZOe9KSsW7cuZ599di6//PJV\nrvavuUINAMBerySvtsXf1LGWvp3EFWoAAFbN+vXr841vfCNJ8tznPjeXXnpp7r///nzlK1/JRz7y\nkZxyyilJJlM+Pv/5z+db3/pWLr300jznOc9ZzbK/jUANAMCqOfLII/ODP/iDeepTn5qPfvSjefrT\nn55nPOMZecELXpALL7wwT3jCE5IkJ598crZu3Zof+IEfyLHHHpsf+7EfS5K86U1vysaNG/PNb34z\nGzduzPnnn/+wPwdTPgAAWFXvfve7v235V3/1Vx8w5jGPeUz+6I/+6AHrL7zwwlx44YX7rLaVcIUa\nAAAGuEINAMCatmXLlmzZsmW1y1jWiq5QV9VpVXVDVe2oqnP3MO7lVdVVddLMuqdX1Uer6rqq+lRV\nHTqPwgEAYC3Y6xXqqlqX5KIkL0qyM8mVVbWtu69fNG59ktcn+fjMuoOTvCvJq7r72qo6Msm9c6wf\nAIAB3b2mvoJuX+jufXr8lVyhPiXJju6+qbvvSXJJkjOXGPeLSd6a5O6ZdS9O8snuvjZJuvv27r5/\nsGYAAObg0EMPze23377PA+dq6u7cfvvtOfTQfTdJovbWwKp6RZLTuvtnpsuvSvKs7t46M+bEJG/u\n7pdX1UKSN3b3VVX1hiTPTPLdSTYkuaS7H/AxzKo6J8k5SbJhw4ZnXnbZZXN5ciS7du3K4Ycfvtpl\nHDD0c370cr70c770c770c37m3cuqymGHHZZ169bN7Zhr0f3335+77rrrAf9wOPXUU6/u7pOW2W3F\nhj+UWFUHJXlbktcsc/znJDk5yTeTfLiqru7uD88O6u6Lk1ycJJs3b+61POl8f7OwsLCmJ/Hvb/Rz\nfvRyvvRzvvRzvvRzfvRybVrJlI9bk2yaWd44Xbfb+iRPTbJQVTcneXaSbdMPJu5M8pHuvq27v5lk\ne5IT51E4AACsBSsJ1FcmOa6qjq2qQ5KclWTb7o3dfWd3P767j+nuY5J8LMkZ3X1Vkg8meVpVfef0\nA4rPT3L9Ax8CAAD2T3sN1N19X5KtmYTjzyS5rLuvq6oLquqMvex7RybTQa5Mck2ST3T3+8fLBgCA\ntWFFc6i7e3sm0zVm1523zNgti5bflclX5wEAwAHHT48DAMAAgRoAAAYI1AAAMECgBgCAAQI1AAAM\nEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAADBCoAQBggEANAAADBGoA\nABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEAYIBADQAAAwRqAAAYIFADAMAAgRoAAAYI\n1AAAMECgBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAA\nDBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEAYIBADQAAAwRq\nAAAYIFADAMAAgRoAAAYI1AAAMGBFgbqqTquqG6pqR1Wdu4dxL6+qrqqTFq3/vqraVVVvHC0YAADW\nkr0G6qpal+SiJKcnOT7J2VV1/BLj1id5fZKPL3GYtyX5wFipAACw9qzkCvUpSXZ0903dfU+SS5Kc\nucS4X0zy1iR3z66sqh9N8vkk1w3WCgAAa85KAvXRSW6ZWd45XfdXqurEJJu6+/2L1h+e5J8n+ZeD\ndQIAwJp08OgBquqgTKZ0vGaJzecn+bXu3lVVezrGOUnOSZINGzZkYWFhtCymdu3apZ9zpJ/zo5fz\npZ/zpZ/zpZ/zo5dr00oC9a1JNs0sb5yu2219kqcmWZiG5ick2VZVZyR5VpJXVNWFSY5I8q2quru7\nf2P2Abr74iQXJ8nmzZt7y5YtD+3Z8AALCwvRz/nRz/nRy/nSz/nSz/nSz/nRy7VpJYH6yiTHVdWx\nmQTps5K8cvfG7r4zyeN3L1fVQpI3dvdVSZ47s/78JLsWh2kAANif7XUOdXffl2Rrkg8m+UySy7r7\nuqq6YHoVGgAAHrFWNIe6u7cn2b5o3XnLjN2yzPrzH2RtAACw5vmlRAAAGCBQAwDAAIEaAAAGCNQA\nADBAoAYAgAECNQAADBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQ\nqAEAYIBADQAAAwRqAAAYIFADAMAAgRoAAAYI1AAAMECgBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAA\nGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAADBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjU\nAAAwQKAGAIABAjUAAAwQqAEAYIBADQAAAwRqAAAYIFADAMAAgRoAAAYI1AAAMECgBgCAAQI1AAAM\nEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBgRYG6qk6rqhuqakdVnbuHcS+vqq6qk6bL\nL6qqq6vqU9M/XzCvwgEAYC04eG8DqmpdkouSvCjJziRXVtW27r5+0bj1SV6f5OMzq29L8rLu/kJV\nPTXJB5McPa/iAQBgta3kCvUpSXZ0903dfU+SS5KcucS4X0zy1iR3717R3f+nu78wXbwuyXdU1aMH\nawYAgDVjr1eoM7mifMvM8s4kz5odUFUnJtnU3e+vqp9f5jgvT/KJ7v7LxRuq6pwk5yTJhg0bsrCw\nsIKyWIldu3bp5xzp5/zo5Xzp53zp53zp5/zo5dq0kkC9R1V1UJK3JXnNHsY8JZOr1y9eant3X5zk\n4iTZvHlzb9myZbQsphYWFqKf86Of86OX86Wf86Wf86Wf86OXa9NKpnzcmmTTzPLG6brd1id5apKF\nqro5ybOTbJv5YOLGJO9L8lPdfeM8igYAgLViJYH6yiTHVdWxVXVIkrOSbNu9sbvv7O7Hd/cx3X1M\nko8lOaO7r6qqI5K8P8m53f2n+6B+AABYVXsN1N19X5KtmXxDx2eSXNbd11XVBVV1xl5235rkyUnO\nq6prprfvHq4aAADWiBXNoe7u7Um2L1p33jJjt8zc/6UkvzRQHwAArGl+KREAAAYI1AAAMECgBgCA\nAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAADBCoAQBggEAN\nAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEAYIBADQAAAwRqAAAYIFADAMAA\ngRoAAAYI1AAAMECgBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBAoAYA\ngAECNQAADBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEAYIBA\nDQAAAwRqAAAYIFADAMAAgRoAAAYI1AAAMECgBgCAASsK1FV1WlXdUFU7qurcPYx7eVV1VZ00s+4X\npvvdUFU/PI+iAQBgrTh4bwOqal2Si5K8KMnOJFdW1bbuvn7RuPVJXp/k4zPrjk9yVpKnJPneJB+q\nqu/v7vvn9xQAAGD1rOQK9SlJdnT3Td19T5JLkpy5xLhfTPLWJHfPrDszySXd/Zfd/fkkO6bHAwCA\nA8Jer1AnOTrJLTPLO5M8a3ZAVZ2YZFN3v7+qfn7Rvh9btO/Rix+gqs5Jck6SbNiwIQsLCysqnr3b\ntWuXfs6Rfs6PXs6Xfs6Xfs6Xfs6PXq5NKwnUe1RVByV5W5LXPNRjdPfFSS5Oks2bN/eWLVtGy2Jq\nYWEh+jk/+jk/ejlf+jlf+jlf+jk/erk2rSRQ35pk08zyxum63dYneWqShapKkick2VZVZ6xgXwAA\n2K+tZA71lUmOq6pjq+qQTD5kuG33xu6+s7sf393HdPcxmUzxOKO7r5qOO6uqHl1VxyY5LskVc38W\nAACwSvZ6hbq776uqrUk+mGRdknd293VVdUGSq7p72x72va6qLktyfZL7krzON3wAAHAgWdEc6u7e\nnmT7onXnLTN2y6LlX07yyw+xPgAAWNP8UiIAAAwQqAEAYIBADQAAAwRqAAAYIFADAMAAgRoAAAYI\n1AAAMECgBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAA\nDBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEAYIBADQAAAwRq\nAAAYIFADAMAAgRoAAAYI1AAAMECgBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAG\nCNQAADBAoAYAgAECNQAADBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUA\nAAwQqAEAYIBADQAAA1YUqKvqtKq6oap2VNW5S2x/bVV9qqquqarLq+r46fpHVdXvTLd9pqp+Yd5P\nAAAAVtNeA3VVrUtyUZLTkxyf5OzdgXnGu7v7ad19QpILk7xtuv7vJnl0dz8tyTOT/GxVHTOn2gEA\nYNWt5Ar1KUl2dPdN3X1PkkuSnDk7oLu/PrN4WJLevSnJYVV1cJLvSHJPktmxAACwXzt4BWOOTnLL\nzPLOJM9aPKiqXpfknyY5JMkLpqv/MJPw/cUk35nkn3T3V0cKBgCAtaS6e88Dql6R5LTu/pnp8quS\nPKu7ty4z/pVJfri7X11VP5jk55K8Jsljk/yvJKd3902L9jknyTlJsmHDhmdedtllQ0+Kv7Zr164c\nfvjhq13GAUM/50cv50s/50s/50s/50cv5+vUU0+9urtPGj3OSq5Q35pk08zyxum65VyS5Den91+Z\n5L91971JvlxVf5rkpCTfFqi7++IkFyfJ5s2be8uWLSsqnr1bWFiIfs6Pfs6PXs6Xfs6Xfs6Xfs6P\nXq5NK5lDfWWS46rq2Ko6JMlZSbbNDqiq42YWX5rkc9P7f57p9I+qOizJs5N8drRoAABYK/Z6hbq7\n76uqrUk+mGRdknd293VVdUGSq7p7W5KtVfXCJPcmuSPJq6e7X5Tkt6rquiSV5Le6+5P74okAAMBq\nWMmUj3T39iTbF607b+b+65fZb1cmX50HAAAHJL+UCAAAAwRqAAAYIFADAMAAgRoAAAYI1AAAMECg\nBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBAoAYAgAECNQAADBCoAQBg\ngEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEAYIBADQAAAwRqAAAYIFAD\nAMAAgRoAAAYI1AAAMECgBgCAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEaAAAGCNQAADBA\noAYAgAECNQAADBCoAQBggEANAAADBGoAABggUAMAwACBGgAABgjUAAAwQKAGAIABAjUAAAwQqAEA\nYIBADQAAAwRqAAAYIFADAMAAgRoAAAasKFBX1WlVdUNV7aiqc5fY/tqq+lRVXVNVl1fV8TPbnl5V\nH62q66ZjDp3nEwAAgNW010BdVeuSXJTk9CTHJzl7NjBPvbu7n9bdJyS5MMnbpvsenORdSV7b3U9J\nsiXJvfMrHwAAVtdKrlCfkmRHd9/U3fckuSTJmbMDuvvrM4uHJenp/Rcn+WR3Xzsdd3t33z9eNgAA\nrA3V3XseUPWKJKd1989Ml1+V5FndvXXRuNcl+adJDknygu7+XFW9Ickzk3x3kg1JLunuC5d4jHOS\nnJMkGzZseOZll102/MSY2LVrVw4//PDVLuOAoZ/zo5fzpZ/zpZ/zpZ/zo5fzdeqpp17d3SeNHufg\neRSTJN19UZKLquqVSd6S5NXT4z8nyclJvpnkw1V1dXd/eNG+Fye5OEmq6hunnnrqDfOqizw+yW2r\nXcQBRD/nRy/nSz/nSz/nSz/nRy/na/M8DrKSQH1rkk0zyxun65ZzSZLfnN7fmeQj3X1bklTV9iQn\nJvnwMvsmyQ3z+JcCE1V1lX7Oj37Oj17Ol37Ol37Ol37Oj17OV1VdNY/jrGQO9ZVJjquqY6vqkCRn\nJdm2qJjjZhZfmuRz0/sfTPK0qvrO6QcUn5/k+vGyAQBgbdjrFeruvq+qtmYSjtcleWd3X1dVFyS5\nqru3JdlaVS/M5Bs87shkuke6+46qelsmobyTbO/u9++j5wIAAA+7Fc2h7u7tSbYvWnfezP3X72Hf\nd2Xy1XkrdfGDGMve6ed86ef86OV86ed86ed86ef86OV8zaWfe/2WDwAAYHl+ehwAAAasWqBewc+Z\nP7qqLp1u/3hVHfPwV7l/qKpNVfU/qur66U+8P2AKTlVtqao7pz8Pf01VnbfUsZioqpur6lPTXj3g\nE8A18e+m5+cnq+rE1ahzrauqzTPn3DVV9fXp99PPjnFu7kFVvbOqvlxVn55Z97iq+pOq+tz0z8cu\ns++rp2M+V1WvfviqXruW6eevVtVnp3+X31dVRyyz7x5fFx6Jlunn+VV168zf6Zcss+8ec8AjzTK9\nvHSmjzdX1TXL7OvcXGS5bLTPXj+7+2G/ZfLhxhuTPCmTH4K5Nsnxi8b8XJJ3TO+fleTS1ah1f7gl\n+Z4kJ07vr0/yf5fo55Ykf7Tate4vtyQ3J3n8Hra/JMkHklSSZyf5+GrXvNZv07/3f5HkiYvWOzf3\n3LfnZfJ1o5+eWXdhknOn989N8tYl9ntckpumfz52ev+xq/18Vvu2TD9fnOTg6f23LtXP6bY9vi48\nEm/L9PP8JG/cy357zQGPtNtSvVy0/d8kOW+Zbc7NB/ZkyWy0r14/V+sK9V5/zny6/DvT+3+Y5Ieq\nqh7GGvcb3f3F7v7E9P43knwmydGrW9UB78wkv9sTH0tyRFV9z2oXtcb9UJIbu/vPVruQ/Ul3fyTJ\nVxetnn19/J0kP7rErj+c5E+6+6vdfUeSP0ly2j4rdD+xVD+7+4+7+77p4scy+b0FVmCZ83MlVpID\nHlH21Mtp/vmJJH/wsBa1H9tDNtonr5+rFaiPTnLLzPLOPDAA/tWY6QvdnUmOfFiq249Np8b8rSQf\nX2Lz366qa6vqA1X1lIe1sP1PJ/njqrq6qs5ZYvtKzmG+3VlZ/n8Gzs0H56ju/uL0/l8kOWqJMc7R\nh+YfZPLu01L29rrAX9s6nULzzmXeUnd+PjjPTfKl7v7cMtudm3uwKBvtk9dPH0o8gFTV4Unek+QN\n3f31RZs/kclb7c9I8utJ/vPDXd9+5jndfWKS05O8rqqet9oF7c9q8qNQZyT5T0tsdm4O6Mn7k76u\naQ6q6s1J7kvy+8sM8bqwMr+Z5G8kOSHJFzOZqsCYs7Pnq9POzWXsKRvN8/VztQL1Sn7O/K/G1ORX\nFr8rye0PS3X7oap6VCYnzO9393sXb+/ur3f3run97UkeVVWPf5jL3G90963TP7+c5H2ZvD05ayXn\nMH/t9CSf6O4vLd7g3HxIvrR7itH0zy8vMcY5+iBU1WuS/EiSvzf9n+wDrOB1gSTd/aXuvr+7v5Xk\n32fpPjk/V2iagX48yaXLjXFuLm2ZbLRPXj9XK1Dv9efMp8u7P1X5iiT/fbkXuUe66dyq/5jkM939\ntmXGPGH3HPSqOiWT//b+gbKEqjqsqtbvvp/JB5Y+vWjYtiQ/VRPPTnLnzFtIPNCyV1ecmw/J7Ovj\nq5P8lyXGfDDJi6vqsdO33F88XcciVXVakjclOaO7v7nMmJW8LpC/Cim7/ViW7tNKcgATL0zy2e7e\nudRG5+bS9pCN9s3r5yp++vIlmXzi8sYkb56uuyCTF7QkOTSTt4d3JLkiyZNWq9a1fkvynEzesvhk\nkmumt5ckeW2S107HbE1yXSafpP5Ykr+z2nWv1Vsmnzq/dnq7bub8nO1nJbloev5+KslJq133Wr0l\nOSyTgPxdM+ucmyvv3x9k8rb5vZnM4/uHmXye5MNJPpfkQ0keNx17UpL/MLPvP5i+hu5I8tOr/VzW\nwm2Zfu7IZL7k7tfP3d8w9b1Jtk/vL/m68Ei/LdPP35u+Ln4yk/DyPYv7OV1+QA54JN+W6uV0/W/v\nfr2cGevc3Hs/l8tG++T10y8lAgDAAB9KBACAAQI1AAAMEKgBAGCAQA0AAAMEagAAGCBQAwDAAIEa\nAAAGCNQAADDg/wOGcHHt7Yu5hAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (12,6))\n",
    "top1 = np.array(wide_n_deep.get_validation_summary(\"Top1Accuracy\"))\n",
    "plt.plot(top1[:,0],top1[:,1],label='top1')\n",
    "plt.xlim(0,top1.shape[0]+10)\n",
    "plt.title(\"top1 accuracy\")\n",
    "plt.grid(True)\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PythonRDD[82] at RDD at PythonRDD.scala:48"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "valPairFeatureRdds.unpersist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "sc.stop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
